MULTILINGUAL LANGUAGE MODELS

Better Quality Pre-training Data and T5 Models for African Languages (2023.emnlp-main)

Challenge:	XLM-R models encode language-sensitive information in each language, allowing them to extract features for downstream tasks and cross-lingual transfer learning.
Approach:	They evaluate how multilingual language models maintain a shared multilingual representation space while still encoding language-sensitive information in each language.
Outcome:	The proposed model can extract features for downstream tasks and cross-lingual transfer learning.

Copied to clipboard

Akintunde Oladipo, Mofetoluwa Adeyemi, Orevaoghene Ahia, Abraham Owodunni, Odunayo Ogundepo, David Adelani, Jimmy Lin

An Unsupervised Multiple-Task and Multiple-Teacher Model for Cross-lingual Named Entity Recognition (2022.acl-long)

Challenge:	Existing web crawls have demonstrated quality issues for low-resource languages . Existing pretraining corpora have numerous quality issues .
Approach:	They propose to audit existing pretraining corpora to understand and rectify quality issues . they pretrain a new T5-based model and evaluate its performance on multiple tasks .
Outcome:	The proposed model outperforms existing pretrained models on four NLP tasks.

Copied to clipboard

Zhuoran Li, Chunming Hu, Xiaohui Guo, Junfan Chen, Wenyi Qin, Richong Zhang

Responsible & Trustworthy NLP

Challenge:	Existing models for named entity recognition only consider the potential transferability between two identical tasks across both domains.
Approach:	They propose to use a similarity metric model to improve cross-lingual named entity recognition task on target domain.
Outcome:	Empirical studies on 7 different languages confirm the effectiveness of the proposed model.

SIB-200: A Simple, Inclusive, and Big Evaluation Dataset for Topic Classification in 200+ Languages and Dialects (2024.eacl-long)

Copied to clipboard

David Adelani, Hannah Liu, Xiaoyu Shen, Nikita Vassilyev, Jesujoba Alabi, Yanke Mao, Haonan Gao, En-Shiun Lee

Information Retrieval

Challenge:	despite progress in building multilingual language models evaluation is limited to a few languages with available datasets . despite this, we create a large-scale open-sourced benchmark dataset for topic classification in 205 languages and dialects to address the lack of evaluation dataset for Natural Language Understanding (NLU).
Approach:	They create a large-scale open-sourced benchmark dataset for topic classification in 205 languages and dialects to address the lack of evaluation dataset for Natural Language Understanding (NLU).
Outcome:	The proposed dataset addresses the lack of evaluation dataset for Natural Language Understanding (NLU) for many languages, it is the first publicly available evaluation dataset.

Cross-lingual Similarity of Multilingual Representations Revisited (2022.aacl-main)

Copied to clipboard

Maksym Del, Mark Fishel

Practical Transformer-based Multilingual Text Classification (2021.naacl-industry)

Challenge:	Similarity indexes like CKA and CCA are not suitable for cross-lingual learning analysis.
Approach:	They propose an alternative that is exempt from the difficulties of CKA/CCA and is good specifically in a cross-lingual context.
Outcome:	The proposed method is exempt from the difficulties of CKA/CCA and is good specifically in a cross-lingual context.

Copied to clipboard

Cindy Wang, Michele Banko

Overlap-based Vocabulary Generation Improves Cross-lingual Transfer Among Related Languages (2022.acl-long)

Challenge:	XNLI does not reflect the data availability and task variety of industry applications.
Approach:	They compare transformer-based text classification methods to multilingual models in five different languages . they use a task- and domain-adaptive pretraining and data augmentation technique .
Outcome:	The proposed methods outperform monolingual models on two tasks in five languages . the results show that practical modifications can improve model performance without labeling .

Copied to clipboard

Vaidehi Patil, Partha Talukdar, Sunita Sarawagi

Low-Resource NLP

Challenge:	Pre-trained multilingual models have shown great potential for zero-shot cross-lingual transfer to low web-resource languages (LRLs).
Approach:	They propose a vocabulary generation algorithm which enhances lexical overlap across related languages by generating a token that increases the representation of LRLs.
Outcome:	The proposed approach improves cross-lingual transfer accuracy without reducing HRL representation and accuracy.

ViClaim: A Multilingual Multilabel Dataset for Automatic Claim Detection in Videos (2025.emnlp-main)

Copied to clipboard

Patrick Giedemann, Pius von Däniken, Jan Milan Deriu, Alvaro Rodrigo, Anselmo Peñas, Mark Cieliebak

Multimodality

Challenge:	Existing efforts in misinformation detection focus on written text, leaving a significant gap in addressing the complexity of spoken text in video transcripts.
Approach:	They propose to annotate video transcripts in three languages and six topics using a custom annotation tool.
Outcome:	The proposed tool shows strong cross-validation performance but challenges for generalization to unseen topics.

XLM-T: Multilingual Language Models in Twitter for Sentiment Analysis and Beyond (2022.lrec-1)

Copied to clipboard

Francesco Barbieri, Luis Espinosa Anke, Jose Camacho-Collados

Sentiment Analysis

Challenge:	Language models are ubiquitous in NLP, but current analyses focus on (multilingual variants of) standard benchmarks and task-specific corpora as multilingual signals.
Approach:	They propose a model to train and evaluate multilingual language models in Twitter using a set of Twitter datasets in eight different languages and a XLM-T model.
Outcome:	The proposed model trains and evaluates multilingual models on Twitter.

Analysis of Multi-Source Language Training in Cross-Lingual Transfer (2024.acl-long)

Copied to clipboard

Seonghoon Lim, Taejun Yun, Jinhyeon Kim, Jihun Choi, Taeuk Kim

Do Multilingual Language Models Think Better in English? (2024.naacl-short)

Challenge:	Existing studies on cross-lingual transfer (XLT) methods address data scarcity problem . cross-linguistic transfer (xLT) techniques are effective at fine-tuning multilingual LMs .
Approach:	They propose to use multiple source languages to improve XLT by fine-tuning multilingual models . they propose to employ arbitrary combinations of source languages for XL to improve performance .
Outcome:	The proposed technique improves performance on language-agnostic or task-specific features by using multiple source languages.

Copied to clipboard

Julen Etxaniz, Gorka Azkune, Aitor Soroa, Oier Lacalle, Mikel Artetxe

AlignFreeze: Navigating the Impact of Realignment on the Layers of Multilingual Models Across Diverse Languages (2025.naacl-short)

Challenge:	Existing studies show that translation-test improves performance of multilingual models by translating the input into English using an external machine translation system.
Approach:	They propose a new approach that leverages the few-shot translation capabilities of multilingual language models.
Outcome:	The proposed approach outperforms direct inference on 5 tasks.

Copied to clipboard

Steve Bakos, David Guzmán, Riddhi More, Kelly Chutong Li, Félix Gaschi, En-Shiun Annie Lee

COSY: COunterfactual SYntax for Cross-Lingual Understanding (2021.acl-long)

Challenge:	Realignment techniques are often employed to enhance cross-lingual transfer in multilingual language models, but can degrade performance in languages that differ significantly from the fine-tuned source language.
Approach:	They propose a method that freezes either the lower half or upper half of the layers during realignment to prevent performance degradation.
Outcome:	The proposed method improves Part-of-Speech (PoS) tagging performance in languages where realignment fails.

Copied to clipboard

Sicheng Yu, Hao Zhang, Yulei Niu, Qianru Sun, Jing Jiang

Syntactic Text Processing

Challenge:	Pre-trained multilingual language models suffer from a large performance gap between source and target languages . e.g., multilingual-BERT models are widely used in cross-lingual tasks .
Approach:	They propose a language-agnostic approach to integrate universal syntax into language models . they use SYntax-aware networks and a COunterfactual training method .
Outcome:	The proposed model achieves state-of-the-art performance on natural language inference and question answering without auxiliary training data.

Unsupervised Multilingual Dense Retrieval via Generative Pseudo Labeling (2024.findings-eacl)

Copied to clipboard

Chao-Wei Huang, Chen-An Li, Tsu-Yuan Hsu, Chen-Yu Hsu, Yun-Nung Chen

Improving Cross-lingual Transfer through Subtree-aware Word Reordering (2023.findings-emnlp)

Challenge:	Existing sparse retrieval methods often yield inferior performance in multilingual retrieval, requiring a large amount of paired data, which is costly.
Approach:	They propose an Unsupervised Multilingual dense Retriever trained without paired data which iteratively improves performance of multilingual retrievers.
Outcome:	The proposed framework outperforms supervised baselines on two benchmark datasets and shows that iterative training improves the performance.

Copied to clipboard

Ofir Arviv, Dmitry Nikolaev, Taelin Karidi, Omri Abend

The Impact of Positional Encodings on Multilingual Compression (2021.emnlp-main)

Challenge:	Recent studies show that multilingual language models are not effective when dealing with less-represented languages.
Approach:	They propose a powerful reordering method that learns word-order patterns conditioned on the syntactic context from a small amount of annotated data.
Outcome:	The proposed method outperforms baselines on a variety of tasks and is effective in both zero-shot and few-shot scenarios.

Copied to clipboard

Vinit Ravishankar, Anders Søgaard

Cross-Lingual Transfer from Related Languages: Treating Low-Resource Maltese as Multilingual Code-Switching (2024.eacl-long)

Challenge:	Several modifications have been proposed to improve monolingual language models, but none of them result in better multilingual models.
Approach:	They propose to add positional encodings to token embeddings to preserve word-order information in a non-autoregressive setting.
Outcome:	The proposed modifications tend to improve monolingual models, but none improve multilingual models.

Copied to clipboard

Kurt Micallef, Nizar Habash, Claudia Borg, Fadhl Eryani, Houda Bouamor

T3L: Translate-and-Test Transfer Learning for Cross-Lingual Text Classification (2023.tacl-1)

Challenge:	Multilingual models exhibit impressive cross-lingual transfer capabilities on unseen languages, but performance is impacted when there is a script disparity with the languages used in the model’s pre-training data.
Approach:	They propose a novel method to align a resource-rich language's script with a target language and train a classifier that can make informed decisions regarding the appropriate processing of each token.
Outcome:	The proposed model can be used to transfer a language's scripts across multiple languages, but it is suboptimal for mixed languages, where only a subset benefits while the rest is impeded.

Copied to clipboard

Inigo Jauregi Unanue, Gholamreza Haffari, Massimo Piccardi

COPAL-ID: Indonesian Language Reasoning with Local Culture and Nuances (2024.naacl-long)

Challenge:	Existing approaches to cross-lingual text classification leverage text classifiers trained in a high-resource language to perform text classification in other languages with no or minimal fine-tuning.
Approach:	They propose to combine a neural machine translator and a text classifier trained in a high-resource language to perform text classification in other languages with no or minimal fine-tuning.
Outcome:	The proposed approach significantly improves over a baseline approach.

Copied to clipboard

Haryo Wibowo, Erland Fuadi, Made Nityasya, Radityo Eko Prasojo, Alham Aji

Reasoning

Challenge:	Existing multilingual language models struggle to capture local nuances and contexts that vary from culture to culture.
Approach:	They propose a public Indonesian language common sense reasoning dataset COPAL-ID . it incorporates Indonesian local and cultural nuances and provides a more natural portrayal of causal reasoning .
Outcome:	The proposed dataset is fluent and free from awkward phrases, unlike the previous dataset.

Multilingual BERT has an accent: Evaluating English influences on fluency in multilingual models (2023.findings-eacl)

Copied to clipboard

Isabel Papadimitriou, Kezia Lopez, Dan Jurafsky

Enhancing Society-Undermining Disinformation Detection through Fine-Grained Sentiment Analysis Pre-Finetuning (2024.findings-eacl)

Challenge:	Multilingual models can improve NLP performance on low-resource languages by leveraging higher-resourced languages, but they also reduce average performance on all languages.
Approach:	They propose a method to evaluate multilingual models by asking if models predict languages with an 'English accent' they propose to use grammatical structure bias to determine if multilingual model is biased toward English-like setting .
Outcome:	The proposed method compares the fluency of multilingual models to the fluencies of monolingual Spanish and Greek models.

Copied to clipboard

Tsung-Hsuan Pan, Chung-Chi Chen, Hen-Hsen Huang, Hsin-Hsi Chen

Tomato, Tomahto, Tomate: Do Multilingual Language Models Understand Based on Subword-Level Semantic Concepts? (2025.findings-naacl)

Challenge:	a new method for disinformation detection is needed to address the issue of disinformation, authors argue . a series of rigorous experiments establishes a notable connection between disinformation and fine-grained sentiment labels .
Approach:	They propose a method leveraging pre-finetuning concept for efficient detection and removal of disinformation that may undermine society.
Outcome:	The proposed method improves performance across languages and languages, showing promising results.

Copied to clipboard

Crystina Zhang, Jing Lu, Vinh Q. Tran, Tal Schuster, Donald Metzler, Jimmy Lin

Kardeş-NLU: Transfer to Low-Resource Languages with Big Brother’s Help – A Benchmark and Evaluation for Turkic Languages (2024.eacl-long)

Challenge:	a recent study shows that human understanding of text depends on general semantic concepts of words that are robust to their superficial forms.
Approach:	They evaluate the accuracy of multilingual multilingual language models based on subword-level semantics . they form "semantic tokens" by merging semantically similar subwords and embeddings based upon the results .
Outcome:	The proposed models are able to make predictions on multilingual tasks with different tokenizers and model sizes.

Copied to clipboard

Lütfi Kerem Senel, Benedikt Ebing, Konul Baghirova, Hinrich Schuetze, Goran Glavaš

Common Sense Beyond English: Evaluating and Improving Multilingual Language Models for Commonsense Reasoning (2021.acl-long)

Challenge:	Cross-lingual transfer (XLT) driven by massively multilingual language models (mmLMs) has been shown to be ineffective for low-resource (LR) target languages with little (or no) representation in mmLM’s pretraining .
Approach:	They propose a benchmark to evaluate cross-lingual transfer (XLT) to LR languages that do have a close HR relative and a framework to integrate Turkish into XLT.
Outcome:	The proposed configuration is of practical relevance for more of the world’s languages: XLT to LR languages that do have a close HR relative.

Copied to clipboard

Bill Yuchen Lin, Seyeon Lee, Xiaoyang Qiao, Xiang Ren

Reasoning

Challenge:	Using multilingual language models, commonsense reasoning research has been limited to English.
Approach:	They propose a Mickey Probe task to evaluate commonsense across languages . they propose X-CSQA and XCODAH datasets to be translated to 14 languages based on the Mickey corpus .
Outcome:	The proposed method significantly improves sentence representations beyond English.

Exploiting Language Relatedness for Low Web-Resource Language Model Adaptation: An Indic Languages Study (2021.acl-long)

Copied to clipboard

Yash Khemchandani, Sarvesh Mehtani, Vaidehi Patil, Abhijeet Awasthi, Partha Talukdar, Sunita Sarawagi

Modular Sentence Encoders: Separating Language Specialization from Cross-Lingual Alignment (2025.acl-long)

Challenge:	Recent research in multilingual language models (LMs) has demonstrated their ability to effectively handle multiple languages in a single model.
Approach:	They propose to exploit relatedness among languages in a language family to overcome corpora limitations of LRLs.
Outcome:	The proposed model exploits relatedness among languages in a language family to overcome corpora limitations for low web-resource languages.

Copied to clipboard

Yongxin Huang, Kexin Wang, Goran Glavaš, Iryna Gurevych

Do We Need Language-Specific Fact-Checking Models? The Case of Chinese (2024.emnlp-main)

Challenge:	Multilingual sentence encoders are often trained to map sentences from different languages into a shared semantic vector space . cross-lingual alignment training distorts optimal monolingual structure of semantic spaces of individual languages . a modular solution can be used for cross-linguistic tasks such as cross-language semantic similarity and zero-shot transfer .
Approach:	They propose a modular training system that embeds sentences from different languages into a shared semantic vector space.
Outcome:	The proposed solution achieves better performance across all tasks compared to monolithic models.

Copied to clipboard

Caiqi Zhang, Zhijiang Guo, Andreas Vlachos

Responsible & Trustworthy NLP

Challenge:	Existing fact-checking models in other languages lack grounding in real-world claims . current models are constrained to a single domain, like COVID-19 .
Approach:	They propose a Chinese document-level evidence retriever that can be translated into Chinese . they then construct an adversarial dataset that is more robust toward biases .
Outcome:	The proposed method outperforms translation-based methods and multilingual language models and is more robust toward biases.

Structural Contrastive Pretraining for Cross-Lingual Comprehension (2023.findings-acl)

Copied to clipboard

Nuo Chen, Linjun Shou, Tengtao Song, Ming Gong, Jian Pei, Jianhui Chang, Daxin Jiang, Jia Li

Speaking Multiple Languages Affects the Moral Bias of Language Models (2023.findings-acl)

Challenge:	Existing methods to train multilingual language models using pretraining tasks like mask language modeling have yielded promising results on a wide range of downstream tasks.
Approach:	They propose a new task to align the structural words in a parallel sentence, enhancing models’ ability to comprehend cross-lingual representations.
Outcome:	The proposed task improves model's ability to comprehend cross-lingual representations by increasing the frequency of negative pairings.

Copied to clipboard

Katharina Haemmerl, Bjoern Deiseroth, Patrick Schramowski, Jindřich Libovický, Constantin Rothkopf, Alexander Fraser, Kristian Kersting

MINERS: Multilingual Language Models as Semantic Retrievers (2024.findings-emnlp)

Challenge:	Pre-trained multilingual language models are often better on English than other languages . however, they are trained on varying amounts of data for each language .
Approach:	They apply the MORALDIRECTION framework to multilingual models and analyse their results . they find that PMLMs encode differing moral biases, but these do not correspond to cultural differences or commonalities in human opinions.
Outcome:	The proposed model captures moral norms from English and imposes them on other languages.

Copied to clipboard

Genta Winata, Ruochen Zhang, David Adelani

Investigating Bias in Multilingual Language Models: Cross-Lingual Transfer of Debiasing Techniques (2023.emnlp-main)

Challenge:	Existing benchmarks have evaluated language models to evaluate their performance across a range of embedding tasks.
Approach:	They propose a benchmark to evaluate the robustness of multilingual language models in semantic retrieval tasks including bitext mining and classification via retrieval-augmented contexts.
Outcome:	The proposed framework evaluates the robustness of multilingual LMs in retrieval tasks across over 200 languages, including extremely low-resource languages in challenging cross-lingual and code-switching settings.

Copied to clipboard

Manon Reusens, Philipp Borchert, Margot Mieskes, Jochen De Weerdt, Bart Baesens

Robustness in NLP

Challenge:	Debiasing techniques that target sentence representations are being investigated in multilingual models . a growing interest in addressing bias detection and mitigation in NLP due to their societal implications.
Approach:	They examine the transferability of debiasing techniques across different languages within multilingual models by using a dataset from CrowS-Pairs.
Outcome:	The proposed techniques reduce bias in English, French, German, and Dutch by 13% . the authors also show that the techniques with additional pretraining exhibit enhanced cross-lingual effectiveness for the languages included in the analyses .

Analyzing the Evaluation of Cross-Lingual Knowledge Transfer in Multilingual Language Models (2024.eacl-long)

Copied to clipboard

Sara Rajaee, Christof Monz

Exploring the Relationship between Alignment and Cross-lingual Transfer in Multilingual Transformers (2023.findings-acl)

Challenge:	Recent advances in training multilingual models on large datasets have shown promising results in knowledge transfer across languages.
Approach:	They challenge the assumption that high zero-shot performance reflects high cross-lingual ability by introducing more challenging setups involving instances with multiple languages.
Outcome:	The proposed model can achieve high performance on multilingual benchmarks and on low-resource languages.

Copied to clipboard

Felix Gaschi, Patricio Cerda, Parisa Rastin, Yannick Toussaint

Rethinking Vocabulary Augmentation: Addressing the Challenges of Low-Resource Languages in Multilingual Models (2025.coling-main)

Challenge:	despite lack of explicit cross-lingual training data, multilingual models can achieve cross-linguistic transfer.
Approach:	They find alignment is significantly correlated with cross-lingual transfer . they advocate for further research on realignment methods for smaller models .
Outcome:	The proposed method outperforms XLM-R Large in POS-tagging between English and Arabic by +15.8 accuracy.

Copied to clipboard

Nankai Lin, Peijian Zeng, Weixiong Zheng, Shengyi Jiang, Dong Zhou, Aimin Yang

PRAM: An End-to-end Prototype-based Representation Alignment Model for Zero-resource Cross-lingual Named Entity Recognition (2023.findings-acl)

Challenge:	Existing methods to augment vocabularies ignore the disparities between model representation and frequency distributions.
Approach:	They propose an Entropy-Consistency Word Selection method which integrates semantic and frequency metrics for vocabulary augmentation.
Outcome:	The proposed method improves performance for low-resource languages compared to high-resourced ones . it integrates semantic and frequency metrics for vocabulary augmentation .

Copied to clipboard

Yucheng Huang, Wenqiang Liu, Xianli Zhang, Jun Lang, Tieliang Gong, Chen Li

Representation Learning

Challenge:	Existing methods to address the named entity recognition problem are limited and lack explicit optimization specific to the task.
Approach:	They propose a prototype-based representation alignment model for a cross-lingual named entity recognition task using labeled source language data.
Outcome:	The proposed model outperforms existing state-of-the-art methods in some challenging scenarios.

Multi-level Distillation of Semantic Knowledge for Pre-training Multilingual Language Model (2022.emnlp-main)

Copied to clipboard

Mingqi Li, Fei Ding, Dan Zhang, Long Cheng, Hongxin Hu, Feng Luo

Translation Quality Estimation by Jointly Learning to Score and Rank (2020.emnlp-main)

Challenge:	Existing methods for improving multilingual models did not focus on learning the semantic structure of representation.
Approach:	They propose a method to improve multilingual language models by aligning parallel sentences . they propose token-, word-, sentence- and structure-level alignment objectives .
Outcome:	The proposed method outperforms baseline models on XNLI, PAWS-X, and XQuAD . it obtains comparable performance on low-resource languages, the authors show .

Copied to clipboard

Jingyi Zhang, Josef van Genabith

MultiTACRED: A Multilingual Version of the TAC Relation Extraction Dataset (2023.acl-long)

Challenge:	The translation quality estimation (QE) task aims to evaluate the general quality of a translation without using reference translations.
Approach:	They propose a translation quality estimation task that uses translations as reference . they propose supervised learning using cross-lingual sentence embeddings from pre-trained multilingual models.
Outcome:	The proposed model outperforms sentBLEU on the WMT 2019 QE as a Metric task and outperformed sentBLUE on the QE in a multilingual language task.

Copied to clipboard

Leonhard Hennig, Philippe Thomas, Sebastian Möller

A Recipe of Parallel Corpora Exploitation for Multilingual Large Language Models (2025.findings-naacl)

Challenge:	Relation extraction (RE) is a fundamental task in information extraction, but its extension to multilingual settings is hindered by the lack of supervised resources comparable in size to large English datasets.
Approach:	They propose a dataset to analyze relation extraction (RE) in multilingual settings . they find machine translation is a viable strategy to transfer RE instances .
Outcome:	The proposed dataset covers 12 typologically diverse languages from 9 language families and is compared with existing datasets.

Copied to clipboard

Peiqin Lin, Andre Martins, Hinrich Schuetze

Structural Priming Demonstrates Abstract Grammatical Representations in Multilingual Language Models (2023.emnlp-main)

Challenge:	Recent studies have highlighted the potential of exploiting parallel corpora to enhance multilingual large language models.
Approach:	They investigate the impact of parallel corpora quality and quantity, training objectives, and model size on performance of multilingual large language models enhanced with parallel corporeal.
Outcome:	The proposed approach improves performance in bilingual and general-purpose tasks.

Copied to clipboard

James Michaelov, Catherine Arnett, Tyler Chang, Ben Bergen

Factual Consistency of Multilingual Pretrained Language Models (2022.findings-acl)

Challenge:	Abstract grammatical knowledge is key to linguistic generalization in humans . strong evidence for grammatikal abstraction in humans comes from structural priming .
Approach:	They compare human models of crosslingual structural priming to human models . they find evidence for abstract monolingual and crosslingual grammatical representations .
Outcome:	The results show that grammatical representations in multilingual models are similar to humans . the strongest evidence for grammatikal abstraction in humans comes from structural priming .

Copied to clipboard

Constanza Fierro, Anders Søgaard

Cross-Align: Modeling Deep Cross-lingual Interactions for Word Alignment (2022.emnlp-main)

Challenge:	Recent work shows that monolingual English language models fill-in-the-blank differently for paraphrases describing the same fact.
Approach:	They propose a resource to analyze consistency of English language models . they find that mBERT is as inconsistent as English BERT in paraphrases .
Outcome:	The proposed model is as inconsistent as English BERT in English paraphrases, but it is more so for all the other 45 languages.

Copied to clipboard

Siyu Lai, Zhen Yang, Fandong Meng, Yufeng Chen, Jinan Xu, Jie Zhou

MergeDistill: Merging Language Models using Pre-trained Distillation (2021.findings-acl)

Challenge:	Existing word alignment models capture few interactions between input sentence pairs, which severely degrades the word alignment quality.
Approach:	They propose to model deep interactions between input and target sentences using a two-stage training framework to train the model.
Outcome:	The proposed model achieves the state-of-the-art (SOTA) performance on four out of five language pairs.

Copied to clipboard

Simran Khanuja, Melvin Johnson, Partha Talukdar

ViGLUE: A Vietnamese General Language Understanding Benchmark and Analysis of Vietnamese Language Models (2024.findings-naacl)

Challenge:	Existing pre-trained multilingual language models often lack capacity and skewed data . this leads to inequitable representation of languages due to limited capacity and sub-optimal vocabularies.
Approach:	They propose a framework to merge pre-trained multilingual language models to maximize their assets with minimal dependencies.
Outcome:	The proposed framework outperforms teacher-trained models on multiple datasets and with a fixed model capacity.

Copied to clipboard

Minh-Nam Tran, Phu-Vinh Nguyen, Long Nguyen, Dien Dinh

When is BERT Multilingual? Isolating Crucial Ingredients for Cross-lingual Transfer (2022.naacl-main)

Challenge:	Existing benchmarks for natural language understanding have been suggested, but there is a lack of such a benchmark in Vietnamese due to the difficulty in accessing datasets or the scarcity of task-specific datasets.
Approach:	They propose to use a benchmark to evaluate Vietnamese language models in a variety of tasks and areas to explore the relationship between specific tasks and the number of shots.
Outcome:	The proposed benchmark contains twelve tasks and encompasses over ten areas and subjects, enabling it to evaluate models comprehensively over a broad spectrum of aspects.

Copied to clipboard

Ameet Deshpande, Partha Talukdar, Karthik Narasimhan

Revisiting non-English Text Simplification: A Unified Multilingual Benchmark (2023.acl-long)

Challenge:	Recent work on multilingual language models has demonstrated their capacity for cross-lingual zero-shot transfer on downstream tasks.
Approach:	They conduct a large-scale empirical study to isolate the effects of various linguistic properties by measuring zero-shot transfer between four different natural languages.
Outcome:	The proposed model exhibits decent cross-lingual zero-shot transfer, with no significant differences in word order and embedding alignment.

Copied to clipboard

Michael Ryan, Tarek Naous, Wei Xu

Paraphrasing

Challenge:	Recent advances in English automatic text simplification have pushed the frontier of multilingual text simulating.
Approach:	They propose to use multilingual evaluation benchmarks to evaluate multilingual text simplification models in English and other languages.
Outcome:	The proposed benchmark outperforms pre-trained models in Russian in zero-shot cross-lingual transfer to low-resource languages.

Adapters for Enhanced Modeling of Multilingual Knowledge and Text (2022.findings-emnlp)

Copied to clipboard

Yifan Hou, Wenxiang Jiao, Meizhen Liu, Carl Allen, Zhaopeng Tu, Mrinmaya Sachan

Wiki-40B: Multilingual Language Model Dataset (2020.lrec-1)

Challenge:	Large language models learn facts from text corpora, but knowledge graphs contain facts in an explicit triple format, restricting their research and application.
Approach:	They propose to enhance multilingual language models with knowledge from multilingual knowledge graphs . they propose to use cross-lingual entity alignment and facts from MLKGs to improve performance .
Outcome:	The proposed model improves MLLMs with cross-lingual entity alignment and facts from multilingual knowledge graphs for many languages while maintaining performance on other general language tasks.

Copied to clipboard

Mandy Guo, Zihang Dai, Denny Vrandečić, Rami Al-Rfou

To Translate or Not to Translate: A Systematic Investigation of Translation-Based Cross-Lingual Transfer to Low-Resource Languages (2024.naacl-long)

Challenge:	We propose a new multilingual language model benchmark that is composed of 40+ languages spanning several scripts and linguistic families.
Approach:	They propose a multilingual language model benchmark composed of 40+ languages . they train monolingual causal language models using a state-of-the-art model .
Outcome:	The proposed model is composed of 40+ languages spanning several scripts and linguistic families.

Copied to clipboard

Benedikt Ebing, Goran Glavaš

Tokenization Impacts Multilingual Language Modeling: Assessing Vocabulary Allocation and Overlap Across Languages (2023.findings-acl)

Challenge:	XLT with multilingual language models is superfluous, says a new study . mBERT, XLM-R and mT5 are effective for cross-lingual transfer, authors say .
Approach:	They propose to use multilingual language models to improve cross-lingual transfer (XLT) they propose to add reliable translations to training data for XLT even for non-MT languages .
Outcome:	The proposed approaches outperform zero-shot XLT with mLMs, the authors show . the authors believe their findings warrant a broader inclusion of more robust translation-based baselines in XL research.

Copied to clipboard

Tomasz Limisiewicz, Jiří Balhar, David Mareček

Syntactic Text Processing

Challenge:	Multilingual language models perform surprisingly well in a variety of NLP tasks for diverse languages.
Approach:	They propose to evaluate the quality of lexical representation and vocabulary overlap observed in sub-word tokenizers.
Outcome:	The proposed criteria show that the overlap of vocabulary across languages can be detrimental to certain downstream tasks.

Correlations between Multilingual Language Model Geometry and Crosslingual Transfer Performance (2024.lrec-main)

Copied to clipboard

Cheril Shah, Yashashree Chandak, Atharv Mahesh Mane, Benjamin Bergen, Tyler A. Chang

CroCoSum: A Benchmark Dataset for Cross-Lingual Code-Switched Summarization (2024.lrec-main)

Challenge:	Pre-trained multilingual language models represent multiple languages in a single vector space, a feature hypothesized to enable impressive crosslingual transfer capabilities.
Approach:	They propose to use a multilingual representation space that sorts axes based on their language-separability to determine whether geometric distances between languages correlate with crosslingual transfer performance.
Outcome:	The proposed measures do not generalize well across models, layers, and tasks.

Copied to clipboard

Ruochen Zhang, Carsten Eickhoff

Cross-Lingual NLU: Mitigating Language-Specific Impact in Embeddings Leveraging Adversarial Learning (2024.lrec-main)

Challenge:	Cross-lingual summarization (CLS) has attracted increasing interest due to the availability of large-scale web-mined datasets and the advancements of multilingual language models.
Approach:	They propose a dataset of cross-lingual code-switched summaries in Chinese and English . they show that leveraging existing CLS resources does not improve performance .
Outcome:	The proposed method does not improve on CroCoSum, indicating the limited generalizability of existing approaches.

Copied to clipboard

Saedeh Tahery, Sahar Kianian, Saeed Farzi

Prompt-Tuning Can Be Much Better Than Fine-Tuning on Cross-lingual Understanding With Multilingual Language Models (2022.findings-emnlp)

Challenge:	Low-resource languages and computational expenses pose significant challenges in the domain of large language models.
Approach:	They propose a novel approach that uses adversarial techniques to mitigate the impact of language-specific information in contextual embeddings generated by large multilingual language models.
Outcome:	The proposed approach excels in zero-shot scenarios for Latin languages like Spanish, but fails to perform for languages distant from English, such as Thai and Persian.

Copied to clipboard

Lifu Tu, Caiming Xiong, Yingbo Zhou

The Norwegian Colossal Corpus: A Text Corpus for Training Large Norwegian Language Models (2022.lrec-1)

Challenge:	Pre-trained multilingual language models show significant performance gains for zero-shot cross-lingual model transfer on a wide range of natural language understanding (NLU) tasks.
Approach:	They do cross-lingual evaluation using prompt tuning and compare it with fine-tuning . prompt tuning achieves much better cross-linguistic transfer than fine- tuning .
Outcome:	The results show that prompt tuning achieves better cross-lingual transfer than fine-tuning across datasets, with only 0.1% to 0.3% tuned parameters.

Copied to clipboard

Per Kummervold, Freddy Wetjen, Javier de la Rosa

SpanAlign: Sentence Alignment Method based on Cross-Language Span Prediction and ILP (2020.coling-main)

Challenge:	Norwegian is one of many languages lacking sufficient textual data to train quality language models.
Approach:	They propose to release 49GB of clean Norwegian textual data containing over 7B words . they hope to foster the creation of better Norwegian language models and multilingual language models .
Outcome:	The Norwegian Colossal Corpus (NCC) contains 49GB of clean Norwegian textual data containing over 7B words.

Copied to clipboard

Katsuki Chousa, Masaaki Nagata, Masaaki Nishino

Exploring Anisotropy and Outliers in Multilingual Language Models for Cross-Lingual Semantic Sentence Similarity (2023.findings-acl)

Challenge:	Existing methods for automatic sentence alignment assume monotonic alignments, but they can handle non-monotonic alignments.
Approach:	They propose a method to automatically extract parallel sentences from noisy parallel documents by embeddings and encoding each source and target sentence.
Outcome:	The proposed method improves translation accuracy by 4.1 BLEU scores on English-Japanese . it can predict spans in target document from sentences in source document .

Copied to clipboard

Katharina Haemmerl, Alina Fastowski, Jindřich Libovický, Alexander Fraser

A Study on Accessing Linguistic Information in Pre-Trained Language Models by Using Prompts (2023.emnlp-main)

Challenge:	Recent studies have shown that contextual language models display outlier dimensions . this is true for monolingual and multilingual models, but little work has been done on multilingual contexts .
Approach:	They investigate outlier dimensions and their relationship to anisotropy in multilingual contexts . they focus on cross-lingual semantic similarity tasks .
Outcome:	The proposed model improves on cross-lingual semantic similarity tasks.

Copied to clipboard

Marion Di Marco, Katharina Hämmerl, Alexander Fraser

Pretraining with Artificial Language: Studying Transferable Knowledge in Language Models (2022.acl-long)

Challenge:	Existing methods to access linguistic information in pre-trained multilingual language models are difficult to use.
Approach:	They propose prompting and formulate linguistic tasks to test the LM's access to explicit grammar principles and find out what type of information can be obtained .
Outcome:	The proposed method can provide access to linguistic features in pre-trained models, but some are harder to capture .

Copied to clipboard

Ryokan Ri, Yoshimasa Tsuruoka

Searching for Needles in a Haystack: On the Role of Incidental Bilingualism in PaLM’s Translation Capability (2023.acl-long)

Challenge:	Existing studies show that pretraining with an artificial language with nesting dependency structure provides some knowledge transferable to natural language.
Approach:	They propose to pretrain artificial languages with structural properties that mimic natural language and then test their performance on downstream tasks.
Outcome:	The proposed language models show strong performance across languages and languages.

Copied to clipboard

Eleftheria Briakou, Colin Cherry, George Foster

AltCLIP: Altering the Language Encoder in CLIP for Extended Language Capabilities (2023.findings-acl)

Challenge:	Large multilingual language models exhibit impressive zero- or few-shot machine translation capabilities, despite never having been explicitly and intentionally exposed to translation data.
Approach:	They propose a mixed-method approach to measure and understand incidental bilingualism at scale using the Pathways Language Model.
Outcome:	The proposed model is exposed to over 30 million translation pairs across at least 44 languages.

Copied to clipboard

Zhongzhi Chen, Guang Liu, Bo-Wen Zhang, Qinghong Yang, Ledell Wu

Information Retrieval

Challenge:	Existing methods to build a strong multilingual multimodal representation model are lacking in good-quality text-image pairs.
Approach:	They propose a method to build a strong multilingual multimodal representation model using English text-image pairs instead of a model from scratch.
Outcome:	The proposed model outperforms the original CLIP model on multilingual multimodal benchmarks.

Are we Estimating or Guesstimating Translation Quality? (2020.acl-main)

Copied to clipboard

Shuo Sun, Francisco Guzmán, Lucia Specia

EthioLLM: Multilingual Large Language Models for Ethiopian Languages with Task Evaluation (2024.lrec-main)

Challenge:	A carefully engineered ensemble of pre-trained multilingual language models won the QE shared task at WMT19.
Approach:	They propose to use pre-trained multilingual language models to train quality estimation for machine translation.
Outcome:	A carefully engineered ensemble of pre-trained language models wins the QE shared task at WMT19.

Copied to clipboard

Atnafu Lambebo Tonja, Israel Abebe Azime, Tadesse Destaw Belay, Mesay Gemeda Yigezu, Moges Ahmed Ah Mehamed, Abinew Ali Ayele, Ebrahim Chekol Jibril, Michael Melese Woldeyohannis, Olga Kolesnikova, Philipp Slusallek, Dietrich Klakow, Seid Muhie Yimam

IndicGenBench: A Multilingual Benchmark to Evaluate Generation Capabilities of LLMs on Indic Languages (2024.acl-long)

Challenge:	Low-resource languages are lagging behind current state-of-the-art (SOTA) developments in the field of NLP due to insufficient resources to train LLMs.
Approach:	They propose to use multilingual large language models for five Ethiopian languages and a benchmark dataset to evaluate their performance.
Outcome:	The proposed models outperform existing models in five Ethiopian languages and a benchmark dataset for various downstream NLP tasks.

Copied to clipboard

Harman Singh, Nitish Gupta, Shikhar Bharadwaj, Dinesh Tewari, Partha Talukdar

Breaking the Curse of Multilinguality with Cross-lingual Expert Language Models (2024.emnlp-main)

Challenge:	IndicGenBench is the largest benchmark for evaluating large language models on user-facing generation tasks across a diverse set of 29 Indic languages .
Approach:	They evaluate large language models on user-facing generation tasks across 29 languages . they use human curation to provide multi-way parallel evaluation data for many under-represented languages a github repository .
Outcome:	IndicGenBench is the largest benchmark for evaluating LLMs on user-facing generation tasks across a diverse set of 29 Indic languages covering 13 scripts and 4 language families.

Copied to clipboard

Terra Blevins, Tomasz Limisiewicz, Suchin Gururangan, Margaret Li, Hila Gonen, Noah Smith, Luke Zettlemoyer

Cross-Lingual Unlearning of Selective Knowledge in Multilingual Language Models (2024.findings-emnlp)

Challenge:	Multilingual language models often underperform monolingual ones due to inter-language competition for model parameters.
Approach:	They propose Cross-lingual Expert Language Models (X-ELM) which mitigates inter-language competition by independently training language models on subsets of the multilingual corpus.
Outcome:	The proposed model outperforms jointly trained multilingual models across all 16 considered languages and transfer the gains to downstream tasks.

Copied to clipboard

Minseok Choi, Kyunghyun Min, Jaegul Choo

Breaking the Language Barrier: Improving Cross-Lingual Reasoning with Structured Self-Attention (2023.findings-emnlp)

Challenge:	Pretrained language models memorize large amounts of information, raising significant safety concerns.
Approach:	They propose an approach to machine unlearning for multilingual language models that selectively erases information across different languages while maintaining overall performance.
Outcome:	The proposed approach is compared with existing unlearning baselines and set a new standard for secure and adaptable multilingual language models.

Copied to clipboard

Negar Foroutan, Mohammadreza Banaei, Karl Aberer, Antoine Bosselut

Understanding Cross-Lingual Alignment—A Survey (2024.findings-acl)

Challenge:	Recent studies show that multilingual language models (MultiLMs) are capable of logically reasoning over natural language statements, reasoning with their implicit knowledge, and performing multi-step reasoning when the model size is large enough.
Approach:	They propose a mechanism that encourages cross-lingual attention in code-switched sequences and improves reasoning performance by up to 14%.
Outcome:	The proposed approach improves reasoning performance by 14% and 4% on the RuleTaker and LeapOfThought datasets.

Copied to clipboard

Katharina Hämmerl, Jindřich Libovický, Alexander Fraser

Wino-X: Multilingual Winograd Schemas for Commonsense Reasoning and Coreference Resolution (2021.emnlp-main)

Challenge:	Cross-lingual alignment is the meaningful similarity of representations across languages in multilingual language models.
Approach:	They propose a taxonomy of methods to improve cross-lingual alignment . they argue that an effective trade-off between language-neutral and language-specific information is key .
Outcome:	The proposed methods can be applied to encoder models and encoder-decoder-only models . they show that language-neutral and language-specific information is key .

Copied to clipboard

Denis Emelin, Rico Sennrich

Cross-Lingual Knowledge Projection and Knowledge Enhancement for Zero-Shot Question Answering in Low-Resource Languages (2025.coling-main)

Challenge:	Winograd schemas are well-established tools for evaluating coreference resolution and commonsense reasoning capabilities of computational models.
Approach:	They present a dataset of German, French, and Russian schemas aligned with their English counterparts.
Outcome:	The proposed model improves in English and German, while the model improve in other languages.

Copied to clipboard

Sello Ralethe, Jan Buys

Unsupervised Cross-lingual Representation Learning at Scale (2020.acl-main)

Challenge:	Knowledge bases (KBs) in low-resource languages are often incomplete, restricting the ability to do zero-shot question answering using multilingual language models.
Approach:	They propose a novel cross-lingual mapping technique which improves word alignments extracted from parallel English-LRL text by combining lexical alignment, named entity recognition, and semantic alignment.
Outcome:	The proposed approach improves zero-shot question answering accuracy by up to 17% compared to baselines without KB access.

Copied to clipboard

Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, Veselin Stoyanov

PMI-Align: Word Alignment With Point-Wise Mutual Information Without Requiring Parallel Training Data (2023.findings-acl)

Challenge:	Pretraining multilingual language models at scale leads to performance gains for cross-lingual transfer tasks.
Approach:	They present a transformer-based multilingual masked language model pre-trained on 100 languages . they show that pretraining multilingual models at scale leads to significant performance gains .
Outcome:	The proposed model outperforms multilingual BERT (mBERT) on cross-lingual benchmarks.

Copied to clipboard

Fatemeh Azadi, Heshaam Faili, Mohammad Javad Dousti

Improving Low-Resource Languages in Pre-Trained Multilingual Language Models (2022.emnlp-main)

Challenge:	Recent studies show that using contextualized embeddings from pre-trained multilingual language models could give us high quality word alignments without the need of parallel training data.
Approach:	They propose a method which uses contextualized embeddings from pre-trained language models to extract word alignments without parallel training.
Outcome:	The proposed method outperforms rival methods on five out of six language pairs.

Copied to clipboard

Viktor Hangya, Hossain Shaikh Saadi, Alexander Fraser

Macedon: Minimizing Representation Coding Rate Reduction for Cross-Lingual Natural Language Understanding (2023.findings-emnlp)

Challenge:	Pre-trained multilingual language models are the foundation of many NLP approaches, but are often not well-supported by these models due to small available monolingual corpora.
Approach:	They propose an unsupervised approach to improve cross-lingual representations of low-resource languages by bootstrapping word translation pairs from monolingual corpora and using them to improve language alignment.
Outcome:	The proposed approach improves cross-lingual representations on low-resource languages using word retrieval and zero-shot named entity recognition.

Copied to clipboard

Haoyu Wang, Yaqing Wang, Huaxiu Yao, Jing Gao

X-SNS: Cross-Lingual Transfer Prediction through Sub-Network Similarity (2023.findings-emnlp)

Challenge:	Existing approaches to learn cross-lingual models require limited data to perform cross-linguistic tasks.
Approach:	They propose a method to remove language-associated information via minimizing representation coding rate reduction.
Outcome:	The proposed model outperforms state-of-the-art models on cross-lingual tasks.

Copied to clipboard

Taejun Yun, Jinhyeon Kim, Deokyeong Kang, Seonghoon Lim, Jihoon Kim, Taeuk Kim

Cross-Lingual Transfer

Challenge:	Cross-lingual transfer (XLT) is an emergent ability of multilingual language models that preserves their performance when evaluated in non-English languages.
Approach:	They propose to use sub-network similarity between two languages as a proxy for XLT prediction.
Outcome:	The proposed method shows proficiency in ranking candidates for zero-shot XLT, achieving an improvement of 4.6% on average in terms of NDCG@3.

XSemPLR: Cross-Lingual Semantic Parsing in Multiple Natural Languages and Meaning Representations (2023.acl-long)

Copied to clipboard

Yusen Zhang, Jun Wang, Zhiguo Wang, Rui Zhang

Multilingual Encoder Knows more than You Realize: Shared Weights Pretraining for Extremely Low-Resource Languages (2025.acl-long)

Challenge:	Existing models for cross-lingual semantic parsing are not able to perform tasks on a wide range of datasets.
Approach:	They propose a benchmark for cross-lingual semantic parsing that uses 22 natural languages and 8 meaning representations to translate queries into MRs.
Outcome:	The proposed benchmarks cover 22 natural languages and 8 meaning representations on 164 domains and 5 tasks covering a wide range of multilingual language models.

Copied to clipboard

Zeli Su, Ziyin Zhang, Guixian Xu, Jianing Liu, Xu Han, Ting Zhang, Yushuang Dong

BenchMAX: A Comprehensive Multilingual Evaluation Suite for Large Language Models (2025.findings-emnlp)

Challenge:	XLM-R and mBART have advanced multilingualism in NLP, but low-resource languages such as Tibetan, Uyghur, Kazakh, and Mongolian are underserved.
Approach:	They propose a framework for adapting multilingual encoders to text generation in extremely low-resource languages by reusing the weights between the encoder and the decoder.
Outcome:	The proposed framework performs better on various downstream tasks even when compared with much larger models.

Copied to clipboard

Xu Huang, Wenhao Zhu, Hanxu Hu, Conghui He, Lei Li, Shujian Huang, Fei Yuan

TUMLU: A Unified and Native Language Understanding Benchmark for Turkic Languages (2025.acl-long)

Challenge:	Existing multilingual benchmarks focus primarily on language understanding tasks.
Approach:	They develop a multi-way multilingual benchmark that measures critical capabilities of large language models across languages.
Outcome:	Extensive experiments on BenchMAX reveal uneven utilization of core capabilities across languages, emphasizing the performance gaps that scaling model size alone does not resolve.

Copied to clipboard

Jafar Isbarov, Arofat Akhundjanova, Mammad Hajili, Kavsar Huseynova, Dmitry Gaynullin, Anar Rzayev, Osman Tursun, Aizirek Turdubaeva, Ilshat Saetov, Rinat Kharisov, Saule Belginova, Ariana Kenbayeva, Amina Alisheva, Abdullatif Köksal, Samir Rustamov, Duygu Ataman

Discriminating Form and Meaning in Multilingual Models with Minimal-Pair ABX Tasks (2025.emnlp-main)

Challenge:	preparing native language MMLU benchmarks is costly and limits representativeness of evaluation datasets.
Approach:	They propose to use a Turkic language MMLU benchmark to assess massive multitask language understanding capabilities.
Outcome:	The proposed benchmarks are based on a Turkic language morphosyntactic and cultural benchmark . the benchmarks evaluate a diverse range of open and proprietary multilingual large language models .

Copied to clipboard

Maureen de Seyssel, Jie Chi, Skyler Seto, Maartje Ter Hoeve, Masha Fedzechkina, Natalie Schluter

Multilingual Definition Modeling (2025.findings-acl)

Challenge:	Existing studies have shown that multilingual models encode languagespecific information and language-agnostic features, but the nature and interaction of these representations is not fully understood.
Approach:	They propose a set of training-free ABX-style discrimination tasks to evaluate how multilingual language models represent language identity (form) and semantic content (meaning).
Outcome:	The proposed tasks show that language discrimination declines over training and strengthens over time and stabilizes in deeper layers.

Copied to clipboard

Edison Marrese-Taylor, Erica K. Shimomoto, A. Solano, Enrique Reid

CommonLID: Re-evaluating State-of-the-Art Language Identification Performance on Web Data (2026.acl-long)

Challenge:	Existing definition modeling tasks are mainly encoder-decoder-based, with no explicit definitions.
Approach:	They propose a multilingual study on definition modeling using monolingual dictionary data for four new languages.
Outcome:	The proposed task is based on monolingual dictionary data for four new languages . results show that multilingual models can perform on-pair with English but cannot leverage potential cross-lingual synergies .

Copied to clipboard

Pedro Ortiz Suarez, Laurie Burchell, Catherine Arnett, Rafael Mosquera, Sara Hincapié Monsalve, Thom Vaughan, Damian Stewart, Malte Ostendorff, Idris Abdulmumin, Vukosi Marivate, Shamsuddeen Hassan Muhammad, Atnafu Lambebo Tonja, Hend Al-Khalifa, Nadia Ghezaiel Hammouda, Verrah Akinyi Otiende, Tack Hwa Wong, Jakhongir Saydaliev, Melika Nobakhtian, Muhammad Ravi Shulthan Habibi, Chalamalasetti Kranti, Carol Muchemi, Khang Nguyen, Faisal Muhammad Adam, Luis Frentzen Salim, Reem Alqifari, Cynthia Jayne Amol, Joseph Marvin Imperial, Ilker Kesen, Ahmad Mustafid, Pavel Stepachev, Leshem Choshen, David Anugraha, Hamada Nayel, Seid Muhie Yimam, Vallerie Alexandra Putra, My Chiffon Nguyen, Azmine Toushik Wasi, Gouthami Vadithya, Rob Van Der Goot, Lanwenn ar C’horr, Karan Dua, Andrew Yates, Mithil Bangera, Yeshil Bangera, Hitesh Laxmichand Patel, Shu Okabe, Fenal Ashokbhai Ilasariya, Dmitry Gaynullin, Genta Indra Winata, Yiyuan Li, Juan Pablo Martínez, Amit Agarwal, Ikhlasul Akmal Hanif, Raia Abu Ahmad, Esther Adenuga, Filbert Aurelian Tjiaranata, Weerayut Buaphet, Michael Anugraha, Sowmya Vajjala, Benjamin L Rice, Azril Hafizi Amirudin, Jesujoba Oluwadara Alabi, Srikant Panda, Yassine Toughrai, Bruhan Kyomuhendo, Daniel Ruffinelli, null Akshata, Manuel Goulão, Ej Zhou, Ingrid Gabriela Franco Ramirez, Cristina Aggazzotti, Konstantin Dobler, Jun Kevin, Quentin Pagès, Nicholas Andrews, Nuhu Ibrahim, Mattes Ruckdeschel, Amr Keleg, Mike Zhang, Casper Rufaro Muziri, Saron Samuel, Sotaro Takeshita, Kun Kerdthaisong, Luca Foppiano, Rasul Dent, Tommaso Green, Ahmad Mustapha Wali, Kamohelo Makaaka, Vicky Feliren, Inshirah Idris, Hande Celikkanat, Abdulhamid Abubakar, Jean Maillard, Benoît Sagot, Thibault Clérice, Kenton Murray, Sarah K. K. Luger

Text Classification

Challenge:	Language identification (LID) is a fundamental step in curating multilingual corpora.
Approach:	They introduce CommonLID, a community-driven, human-annotated LID benchmark for the web domain, covering 109 languages.
Outcome:	The proposed benchmark covers 109 languages and shows that existing evaluations overestimate accuracy for many languages in the web domain.

Just Go Parallel: Improving the Multilingual Capabilities of Large Language Models (2025.acl-long)

Copied to clipboard

Muhammad Reza Qorib, Junyi Li, Hwee Tou Ng

EuroGEST: Investigating gender stereotypes in multilingual language models (2025.emnlp-main)

Challenge:	Large language models (LLMs) have impressive translation capabilities even without being explicitly trained on parallel data.
Approach:	They propose to add parallel data to enhance multilingual encoder-based and encoder decoder language models by focusing on translation and multilingual common-sense reasoning.
Outcome:	The proposed methods show that adding parallel data can significantly improve LLMs’ multilingual capabilities.

Copied to clipboard

Jacqueline Rowe, Mateusz Klimaszewski, Liane Guillou, Shannon Vallor, Alexandra Birch

One Script Instead of Hundreds? On Pretraining Romanized Encoder Language Models (2026.findings-acl)

Challenge:	Large language models encode social biases, but most benchmarks for gender bias remain English-centric.
Approach:	They propose a dataset to measure gender-stereotypical reasoning in large language models across English and 29 European languages.
Outcome:	The proposed method is highly accurate across languages and strong in translations and gender labels.

Copied to clipboard

Benedikt Ebing, Lennart Keller, Goran Glavaš